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ABSTRACT 

The  main  purpose  of  the  study  was  to  determine  reproducibility  limits  of  night  vision  goggle  (NVG)  acuity  measurement 
under  relatively  high  and  low  illumination  levels.  Psychometric  acuity  functions  of  three  observers  were  repeatedly 
measured  using  Landolt  C’s  as  stimuli.  The  reproducibility  limits  of  the  Snellen  acuity  value  (20/xx)  relating  to  50,  75  and 
95%  correct  (adjusted  for  chance)  were  then  determined.  Reproducibility  limit  is  defined  as  approximately  95%  of  all  pairs 
of  replications  (20/xx)  from  the  same  illuminance  and  same  observer,  generated  on  different  days,  should  differ  in  absolute 
value  by  less  than  the  reproducibility  limit.  It  was  determined  that  for  the  lower  illumination  (8.61E-4  lux)  at  50%  corrected 
for  chance  probability  level,  the  reproducibility  limit  was  5.1  Snellen  acuity  (20/xx)  and  for  the  higher  illumination  (1.38E- 
2  lux),  2.5  Snellen  acuity.  These  limits  were  17%  and  13%  of  mean  acuity,  respectively. 

INTRODUCTION  and  BACKGROUND 

There  are  numerous  methods  used  to  determine  night  vision  goggle  (NVG)  visual  acuity  (Pinkus  &  Task,  1998);  limiting 
resolution  (Stefanik,  1994;  Task,  1979),  Snellen  Acuity  (Bailey  &  Lovie,  1979;  Wiley,  1989;  Miller,  Provines,  Block  & 
Tredici,  1984),  square-wave  targets  (Task  &  Genco,  1986),  Landolt  C’s  (Pinkus  &  Task,  1997),  adaptive  psychophysical 
(Simpson,  1989)  and  directly  measuring  the  psychometric  function  (Pinkus  &  Task,  1997;  Brown,  Galanter,  Hess  & 
Mandler,  1962).  Each  method  produces  a  number  that  is  composed  of  the  actual  acuity  value  plus  error.  There  can  be  many 
sources  of  error  but  the  largest  are  the  method  itself  and  the  inherent  variability  of  the  observer  while  working  under 
threshold  conditions.  Observer  variability  is  reduced  to  a  minimum  through  extensive  training,  testing  the  same  time 
everyday  and  shortened  sessions  in  order  to  reduce  eye  fatigue.  Additionally,  even  though  observers  are  given  specific 
instructions,  response  criteria  also  vary  among  or  within  observers;  even  over  the  course  of  a  single  experimental  session. 
To  eliminate  the  criteria  problem,  Pinkus  &  Task  (1998)  used  Landolt  C’s  in  a  four-alternative  forced-choice  (4AFC) 
paradigm  to  measure  the  entire  psychometric  function.  This  paradigm  allowed  for  any  desired  response  criterion  level  (e.g., 
50%  or  75%  corrected  for  chance,  probability  of  detection)  to  be  selected  for  the  prediction  of  NVG  visual  acuity 
performance. 

The  goal  in  this  study  was  to  select  a  stable  method  and  then  determine  its  reproducibility  (ASTM  Practice  E  691). 
Reproducibility  represents  observation-to-observation  variability  under  a  given  set  of  viewing  conditions.  Directly 
measuring  the  psychometric  function  should  keep  variability  due  to  the  test  method  to  a  minimum.  Determining 
reproducibility  will  allow  the  use  of  this  test  method  to  determine  NVG  visual  acuity  with  a  known  error  tolerance.  The 
interpretation  of  visual  acuity  data  is  investigated  in  the  discussion  section. 

METHOD 

Participants 

The  trained  observers  were  one  female  and  two  males  highly  experienced  with  the  operation  of  NVGs.  They  ranged  in  age 
from  37  to  47  years,  each  having  normal  (20/20)  or  corrected-to-normal  binocular  visual  acuity. 

Apparatus  and  Stimuli 

NVGs  -  Participants  viewed  the  target  stimuli  using  a  pair  of  ITT  model  F4949D  (SN  3872)  NVGs.  The  goggles  had  a  gain 
of  approximately  5600  as  measured  using  the  Hoffman  (Stamford,  CT)  ANV-120  NVG  Test  Set.  Before  the  start  of  each 
test  session,  the  optical  alignment  of  the  NVGs  was  verified  using  the  Hoffman  ANV-126  Night  Vision  Tester. 
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Each  test  session  was  conducted  in  a  light-tight  laboratory.  The  observer  was  seated  at  an  optical  table  with  the  NVGs 
secured  in  a  stationary  mount  directly  in  front  of  them.  The  observer  was  able  to  adjust  their  seat  to  the  proper  height  for 
viewing  through  the  NVGs.  An  external  regulated  power  supply  was  used  to  energize  the  goggles. 

At  the  beginning  of  each  test  session  the  observer  would  set  up  and  pre-focus  the  NVGs  using  the  following  procedure. 
After  dark-adapting  for  15  minutes,  the  NVGs  were  powered  on  and  the  observer  adjusted  the  inter-pupillary  distance  until 
a  fused  circular  image  was  visible.  The  observer  (using  their  dominant  eye)  focused  one  channel  at  a  time.  The  observer 
first  focused  on  the  green  scintillation  by  looking  at  the  ceiling  and  adjusting  the  eyepiece  until  the  focus  of  the  scintillation 
was  as  sharp  as  possible.  Next,  the  observer  focused  the  corresponding  objective  lens  by  viewing  the  3  x  3A,  NVG  high- 
contrast  square-wave  resolution  chart  (Task  &  Genco,  1986)  located  at  a  30  ft  distance  (optical  infinity).  The  objective 
lenses  were  pre-focused  using  the  highest  illumination  level  (1.38  x  10-2  lux).  The  observer  chose  the  finest  grating  clearly 
resolvable  and  adjusted  the  objective  lens  to  the  sharpest  focus  possible.  If  necessary,  the  observer  fine  tuned  the  eyepiece 
while  viewing  the  chart.  These  steps  were  repeated  for  the  second  channel. 

Illumination  source  and  Illumination  levels  -  Target  stimuli  were  illuminated  by  one  or  two  moon  illumination  lamps 
outfitted  with  adjustable  2856K  color  temperature  incandescent  bulbs  (MIL-L-85762A).  Metal  apertures  were  used  to 
achieve  the  two  illumination  levels.  Using  apertures  to  vary  illumination  intensity  did  not  affect  the  2856K  color 
temperature.  Table  1  shows  the  illumination  levels  and  corresponding  NVG  eyepiece  luminance  outputs.  The  lowest 
illumination  level  is  approximately  equivalent  to  l/300th  full  moon  (RCA,  1974;  p.  65)  while  the  highest  illumination  level 
is  1 6  times  brighter. 

Table  1.  The  two  illumination  levels  and  corresponding  NVG  output  luminances  for  this  study. 


ILLUMINATION 
ON  LANDOLT  C 

NVG  OUTPUT 
LUMINANCE 

8.61xl0'4  lux 
(8.00xl0"s  fc) 

0.356  nit 
(0.104  fL) 

1.38xl0‘-  lux 
(1.28xl0"3  fc) 

4.324  nit 
(1.262  fL) 

Landolt  C  test  stimuli  and  automated  data  recording  device  -  The  test  stimuli  were  closely-sized  computer-  created,  high 
contrast  (67%  Michelson;  Farrell  &  Booth,  1984)  Landolt  C’s  (National  Academy  of  Sciences,  1980)  printed  using  a  high 
resolution,  photo-grade  laser  printer.  The  print  out  of  each  target  was  mounted  on  18  cm  x  18  cm  (7”  x  7”)  squares  of  foam 
board.  Each  target  varied  in  gap  size  and  represented,  when  converted,  a  specific  Snellen  visual  acuity  value  (20/xx).  The 
back  of  each  target  was  labeled  with  four  different  bar  code  patterns.  Each  bar  code  contained  identification  information  for 
that  particular  target  such  as  target  number,  target  type,  the  corresponding  visual  acuity  (20/xx),  the  target  contrast,  and  the 
gap’s  orientation.  For  each  experimental  trial,  a  Landolt  C  was  placed  in  the  center  of  a  larger  foam  board  surround  56  cm 
x  56  cm  (22”  H  x  22”  L).  This  surround  was  secured  to  the  front  of  a  black  light-tight  wooden  box.  The  box  measured  66 
cm  FI  x  56  cm  W  x  36  cm  L  (26  H  x  22”  W  x  14”  L)  and  sat  on  top  of  a  stand.  The  surround  had  the  same  reflectance  as  the 
background  of  the  Landolt  C’s.  This  box  housed  a  bar  code  scanner/reader  used  to  automate  the  recording  of  Landolt  C 
target  information.  The  light-tight  box  prevented  the  incompatible  red  laser  beam  from  the  bar  code  scanner  from  affecting 
the  NVGs.  The  bar  code  reader  connected  directly  to  a  computer  at  the  experimenter’s  station.  The  entire  set  up  was 
positioned  at  30  feet  or  optical  infinity  from  the  observer.  A  four  button  response  box  located  next  to  the  observer  was  also 
connected  to  the  computer.  The  observer  used  the  buttons  to  indicate  the  orientation  of  the  Landolt  C  gap  (up,  down,  left  or 
right).  The  computer  recorded  the  button  press  response  and  Landolt  C  bar  code  information  as  well  as  other  pertinent 
information. 

Procedure 

Three  trained  observers  participated  in  this  study.  Each  observer  completed  2  sessions  per  day  on  each  of  3  days.  Each 
session  (140  randomized  trials)  used  an  illumination  level  of  either  8.61E-04  or  1.38E-2  lux  (target  background 
reflectance’s  were  5.64E-5  and  9.03E-4  lux,  respectively).  At  the  beginning  of  each  test  session,  the  observer  dark-adapted 
for  approximately  15  minutes.  The  observer  then  turned  on  and  focused  the  NVGs.  For  each  trial,  the  experimenter,  using 
pre-determined  randomized  stimuli  ordering,  placed  a  Landolt  C  onto  a  small  ledge  on  the  surround  and  kept  it  blocked 
from  the  observer’s  view.  This  ledge  centered  the  ‘C’  and  was  not  noticeably  visible  when  viewed  through  the  NVGs.  The 
experimenter  pressed  a  switch  to  scan  the  bar  code  on  the  back  of  the  target.  The  experimenter  would  then  move  away  from 
the  Landolt  C  and  the  observer  had  about  four  seconds  to  view  the  stimulus.  At  the  end  of  the  four-second  interval  the 
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computer  would  beep  an  alarm  and  the  experimenter  would  immediately  block  the  stimulus  from  the  observer’s  view.  The 
observer  would  press  the  appropriate  button  on  the  response  box  to  indicate  what  direction  the  gap  was  oriented:  up,  down, 
left  or  right.  The  observer  was  not  provided  with  any  feedback  on  their  performance.  This  sequence  of  events  was  repeated 
until  140  trials  were  completed  (7  Landolt  C  target  sizes  x  4  orientations  x  5  repetitions  of  each  orientation).  The  7  target 
sizes  are  Snellen  acuity  values  (i.e.,  20/xx). 

RESULTS 

The  percent  of  correctly  identified  orientations  out  of  the  20  trials  (4  orientations  x  5  repetitions)  for  each  combination  of 
illuminance,  observer,  acuity  and  replication  (i.e.,  day)  was  determined.  Chance  alone  would  result  in  25%  correctly 
identified  trials.  It  is  assumed  that  percents  that  are  less  than  25%  would  approach  25%  with  a  sufficient  number  of  trials. 
The  percents  were  transformed  to  adjust  for  chance.  The  procedure  for  this  transformation  was  as  follows: 

Let:  P  =  percent  correct  trials 

Pa  =  percent  correct  trials  adjusted  for  chance 

(1)  if  P  <  25  then  P  =  25 

(2)  PA  =  (P-25)  *  100/75 

For  probit  analysis,  adjusted  percents  are  converted  to  normal  equivalent  deviates  (NED).  An  NED  is  the  value  of  a 
standard  normal  variable  whose  cumulative  probability  (expressed  as  a  percent)  would  equal  the  percent  correct  adjusted 
for  chance.  The  NED  values  are  then  used  as  the  dependent  variable  in  a  linear  regression  with  acuity  as  the  independent 
variable,  where  a  linear  relationship  is  assumed  (Finney,  1980).  The  estimated  NED=bo+bi*acuity  is  then  transformed  back 
to  percents.  The  resulting  estimates  of  Pa  form  a  curvilinear  function. 

If  there  is  a  range  of  acuity  values  where  PA  is  near  0  or  PA  is  near  100,  the  relationship  between  NED  and  acuity  will  not  be 
linear.  The  rationale  for  selecting  percents  used  for  modeling  was  to  start  with  the  largest  acuity  value  where  the  observer 
was  guessing  (correct  <  7  out  of  20  was  used)  if  applicable,  and  end  with  the  smallest  acuity  value  where  PA=100,  if 
applicable.  Since  an  NED  cannot  be  computed  for  0%  or  100%,  0%  was  set  to  1%  and  100%  was  set  to  99%.  Correct  <  7 
was  used  as  a  ‘guessing’  cutoff  since  the  probability  of  8  <  Correct  is  0.10  by  chance  alone.  For  each  illuminance,  observer, 
and  replication  (i.e.,  day),  the  acuity  that  related  to  predicted  50,  75,  and  95%  correct  adjusted  for  chance  was  determined. 
The  reproducibility  limits  (RL)  of  the  acuity  value  (20/xx)  relating  to  50,  75,  and  95%  correct  adjusted  for  chance  were 
determined  and  are  shown  in  table  2. 

Table  2.  Snellen  acuity  reproducibility  limits  (20/xx)  for  each  illuminance. 


Illuminance 

%  Correct 

Mean  Acuity 

RL 

RL  %  of 

(lux) 

(Adjusted) 

(20/xx) 

(20/xx) 

Mean 

50 

29.8 

5.1 

17 

8.61E-04 

75 

33.3 

6.1 

18 

95 

38.5 

9.4 

24 

50 

18.9 

2.5 

13 

1.38E-02 

75 

21.1 

4.5 

21 

95 

24.1 

7.8 

32 

The  same  procedure  for  determining  the  predicted  percent  correct  adjusted  for  chance  that  was  performed  for  each 
illuminance,  observer,  and  replication  was  also  performed  for  each  illuminance  and  observer,  summed  across  replications. 
Figure  1  contains  plots  of  the  predicted  percents.  This  figure  is  provided  to  graphically  show  the  results  of  probit  analysis, 
and  to  demonstrate  differences  among  the  three  observers. 
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lllum=8.61  E-4  lux  Obs  #1  lllum=8.61  E-4  lux  Obs  #2  lllum=8.61E-4  lux  Obs  #3 


Snellen  Acuity  (20/xx) 

Figure  1.  Predicted  percent  correct  adjusted  for  chance,  for  each  illuminance  and  observer. 
Acuity  values  are  given  that  correspond  to  50  and  75%  correct  adjusted  for  chance. 


DISCUSSION 

Table  2  summarizes  the  Snellen  acuity  reproducibility  limits  at  the  two  illumination  levels  for  three  levels  of  percent  correct 
(adjusted  for  chance).  The  50,  75  and  95%  levels  represent  threshold,  a  just-noticeable-difference  (JND)  above  threshold 
and  a  conservative,  high-confidence  gap  detection,  respectively.  The  overall  reproducibility  limits  show  a  higher  variability 
at  the  lower  illuminance  condition. 

One  way  to  apply  the  reproducibility  limit  is  to  use  it  to  assign  a  range  for  a  given  NVG  visual  acuity  measurement.  Many 
times,  the  interactions  between  NVGs  and  other  cockpit  subsystems  such  as  a  gold-coated  canopy  or  an  incompatible  light 
source,  are  evaluated  by  making  only  a  few  NVG  visual  acuity  measurements  (baseline  versus  test  condition)  using  just  a 
couple  of  observers.  The  problem  with  this  approach  is  that  the  variability  that  is  inherent  in  both  the  test  method  and  the 
observer  can  easily  mask  the  true  NVG  acuity  effects.  The  reproducibility  limits  at  the  50%  correct  were  approximately 
15%  of  the  mean  acuity  values.  When  interpreting  field  data  acuity  results,  95%  of  all  absolute  differences  in  observations 
should  differ  by  less  than  15%  of  the  mean  acuity  if  variability  is  due  solely  to  experimental  error.  If  the  data  varies  more 
than  15%  than  there  are  additional  sources  of  variability  such  as  the  infrared-attenuating  effects  of  canopy  coating  on  NVG 
visual  acuity.  For  example,  if,  under  low  light  levels,  an  observer  reports  an  NVG  visual  acuity  baseline  of  20/30  and  then 
20/35  while  looking  through  a  coated  canopy,  one  might  (erroneously)  conclude  that  the  canopy  caused  a  large  loss  of 
acuity  which  translates  into  a  loss  of  target  acquisition  slant  range.  The  difference  between  20/30  and  20/35  is  5  which 
when  divided  by  their  average  of  32.5  equals  0.15  or  15%.  A  difference  of  five  then  is  approximately  at  the  95  percentile  of 
acuity  differences  under  the  same  viewing  conditions.  Therefore,  this  example  data  is  at  the  outer  limit  of  variability  due 
solely  to  experimental  error  implying  that  small  differences  found  in  field  measurements  are  questionable.  The 
reproducibility  value  of  15%  of  the  mean  acuity  value  is  a  good  estimate  for  field  tests.  If  the  tests  are  conducted  under 
more  conservative  criteria  (75%  correct),  then  about  20%  applies. 


CONCLUSIONS 

One  problem  we  have  observed  in  this  and  other  studies  is  that  of  interpretability.  Observers  reliably  report  seeing  Landolt 
C  gaps  at  relatively  high  visual  acuity’s  of  20/20  and  better,  even  under  degraded  viewing  conditions  such  as  low-light 
levels  or  noisy  (scintillation)  conditions.  But  does  the  observer  really  see  the  gap?  When  working  at  noisy  threshold  levels 
the  gap  can  appear  to  move  around  to  the  four  different  locations  or  alternately  open  and  close.  Sometimes  no  gap  is  seen  at 
all,  maybe  just  a  lighter  area  or  a  circle  having  a  flat  side.  When  the  observer  correctly  responds  to  this  flat  side,  we 
interpret  the  correct  response  as  a  seen  gap  having  a  specific  size  indicating  a  specific  visual  acuity  which  explains  the 
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higher  than  expected  visual  acuity’s.  This  problem  underlies  all  acuity  measurements  thus  affecting  interpretability  and 
conclusions  of  studies’  findings.  To  try  to  reduce  this  effect  the  stimulus  duration  was  held  to  four  seconds.  The  current 
method  of  stimulus  presentation  makes  shorter,  precise  duration’s  difficult.  A  tachistoscope-type  apparatus  suited  for  NVG 
optics  will  have  to  be  designed  for  the  next  study  that  will  examine  the  effects  of  short-duration  stimulus  presentations  on 
NVG  acuity. 
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